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Abstract 

Background: Coronary computed tomography angiography has become the foremost noninvasive imaging 
modality of the coronary arteries and is used as an alternative to the reference standard, conventional coronary 
angiography, for direct visualization and detection of coronary artery stenoses in patients with suspected coronary 
artery disease. Nevertheless, there is considerable debate regarding the optimal target population to maximize 
clinical performance and patient benefit. The most obvious indication for noninvasive coronary computed 
tomography angiography in patients with suspected coronary artery disease would be to reliably exclude 
significant stenosis and, thus, avoid unnecessary invasive conventional coronary angiography. To do this, a test 
should have, at clinically appropriate pretest likelihoods, minimal false-negative outcomes resulting in a high 
negative predictive value. However, little is known about the influence of patient characteristics on the clinical 
predictive values of coronary computed tomography angiography. Previous regular systematic reviews and meta- 
analyses had to rely on limited summary patient cohort data offered by primary studies. Performing an individual 
patient data meta-analysis will enable a much more detailed and powerful analysis and thus increase 
representativeness and generalizability of the results. The individual patient data meta-analysis is registered with the 
PROSPERO database (CoMe-CCT, CRD4201 2002780). 
(Continued on next page) 
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Methods/Design: The analysis will include individual patient data from published and unpublished prospective 
diagnostic accuracy studies comparing coronary computed tomography angiography with conventional coronary 
angiography. These studies will be identified performing a systematic search in several electronic databases. 
Corresponding authors will be contacted and asked to provide obligatory and additional data. Risk factors, previous 
test results and symptoms of individual patients will be used to estimate the pretest likelihood of coronary artery 
disease. A bivariate random-effects model will be used to calculate pooled mean negative and positive predictive 
values as well as sensitivity and specificity. The primary outcome of interest will be positive and negative predictive 
values of coronary computed tomography angiography for the presence of coronary artery disease as a function of 
pretest likelihood of coronary artery disease, analyzed by meta-regression. As a secondary endpoint, factors that 
may influence the diagnostic performance and clinical value of computed tomography, such as heart rate and 
body mass index of patients, number of detector rows, and administration of beta blockade and nitroglycerin, will 
be investigated by integrating them as further covariates into the bivariate random-effects model. 

Discussion: This collaborative individual patient data meta-analysis should provide answers to the pivotal question 
of which patients benefit most from noninvasive coronary computed tomography angiography and thus help to 
adequately select the right patients for this test. 

Keywords: Collaborative meta-analysis on cardiac CT, CoMe-CCT, Coronary CT angiography, Individual patient data 
meta-analysis, IPD, Positive and negative predictive value, Pretest likelihood, Sensitivity and specificity, Study 
protocol 



Background 

Coronary artery disease (CAD) is the main cause of 
death in industrialized countries [1]. To allow early 
detection as well as accurately ruling out CAD, a reliable 
noninvasive test would have important advantages. To 
reliably exclude disease, a diagnostic test with a minimal 
proportion of false-negative results and thus a relatively 
high sensitivity should be used [2], Coronary computed 
tomography (CT) angiography has already been evaluated 
comprehensively concerning its diagnostic accuracy 
potentials by single studies and meta-analyses. However, 
the influence of the clinical presentation, risk factors and 
prior test results of patients, which can be used to estimate 
the influence of the pretest likelihood of CAD on the posi- 
tive and negative predictive values of coronary CT angiog- 
raphy, are not fully understood. A few single-center studies 
have been published dealing with the influence of the 
pretest likelihood of CAD on the diagnostic performance of 
CT. Meijboom et al. demonstrated in 254 symptomatic 
patients that the accuracy of coronary CT angiography was 
greatest in patients with low-to-intermediate pretest likeli- 
hood whereas the negative predictive value was substan- 
tially reduced in patients with high pretest likelihood [3]. 
This reduction of the negative predictive values in patients 
with high disease prevalence was also shown in a compar- 
able study by Husmann et al. comprising 88 patients [4]. In 
90 patients, Leber et al. also demonstrated that CT allows 
CAD to be accurately ruled out if an intermediate pretest 
likelihood is present [5]. Applying Bayesian analysis, Dewey 
et al. [6] confirmed that the diagnostic performance of both 
CT and magnetic resonance coronary angiography was 
influenced by the pretest likelihood. Pretest likelihood of 



CAD can be estimated according to clinical presentation 
and risk factors applying several tabulation tools [7-11] and 
the appropriateness of further testing may be estimated 
based on their results. Establishing and further elucidating 
the potential of pretest likelihood estimations to ensure 
the appropriateness of CT is an important aim of cardio- 
vascular medicine. In a meta-regression analysis using 
disease prevalence as a surrogate marker for pretest likeli- 
hood, Schlattmann et al. further specified the boundaries 
of low-to-intermediate pretest likelihood for patients in 
whom coronary CT angiography can be applied as a reliable 
rule-out test [12] (Figure 1). 

The analysis is based on summary patient collective 
data from primary study articles aggregated in a regular 
meta-analysis [13] and, thus, further specific patient- level 
information for more detailed subanalyses were not 
available. 

The purpose and main aim of this project is thus to 
summarize the published and unpublished evidence on 
coronary CT angiography using IPD in a collaborative 
meta-analysis to draw more reliable and generalizable 
conclusions [14] concerning the appropriateness of CT 
in patients with varying presentations and thus pretest 
likelihoods [15]. The results should enable us to determine 
which patients benefit most from noninvasive coronary 
angiography and those less suited to undergo CT. 

Moreover, the influence of additional patient and 
acquisition characteristics will also be summarized in 
our collaborative meta-analysis to allow us to draw 
important conclusions concerning the eligibility of patients 
for coronary CT angiography and the importance of specific 
technical requirements and methods of preparation. 
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Figure 1 Predictive values of coronary computed tomography angiography. The population-averaged predictive values (based on 
published studies and their summary results) are shown as blue (positive) and green (negative) curves with their 95% confidence intervals based 
on the published studies (circles). The negative predictive values are shown as 1 -negative predictive values to avoid overlap with the positive 
predictive value curve. The number of patients included in the published studies is indicated by the size of the circles. The dashed lines indicate 
the predefined minimal predictive values on the y-axis (90% for negative and 70% for the positive predictive values). Such analyses are limited 
because only the published predictive values in the studies and the calculated prevalences (but not the pretest likelihoods [7-9]) are available. 
Thus, individual patient data (IPD) from a collaborative meta-analysis can overcome these limitations and draw more meaningful conclusions 
about the clinical utility of diagnostic tests. Image reproduced from: Schlattmann P, Schuetz GM, Dewey M: Influence of coronary artery disease 
prevalence on predictive values of coronary CT angiography: a meta-regression analysis. Eur Radiol 2011, 21 (9):1 904-1 91 3 [12]. 



Among these factors being discussed or having previously 
been shown to influence the diagnostic performance of 
coronary CT angiography is the heart rate of the patients 
during scanning, which may lead to motion artifacts, and 
the body mass index, which, if severely increased, can 
result in rather noisy nondiagnostic test results [16-18]. 
Also, technical factors such as the number of simultaneous 
CT detector rows and the administration of beta blockade 
to reduce heart rate and nitroglycerin to dilate the vessels 
have been shown in single-center studies to alter the 
diagnostic performance and image quality of coronary 
CT angiography [19-24], In detail, subanalyses will be 
performed to investigate the following questions: 

♦ Is there a decrease in diagnostic accuracy of coronary 
CT angiography in patients with a body mass index >30? 

♦ Does heart rate control (heart rate <60 beats per 
minute) lead to an increase of accuracy? 

♦ Does the use of >64-row CTs lead to an increase of 
diagnostic accuracy? 

♦ Is there a difference in diagnostic accuracy between 
men and women after adjusting for differences in 
pretest likelihood? 



♦ Is diagnostic accuracy higher in patients >70 years of 
age or younger patients after adjusting for differences 
in calcium score? 

♦ Is a certain Agatston calcium score threshhold a good 
universal predictor for coronary artery disease? 

♦ Is diagnostic accuracy of CT different in patients with 
atypical or typical chest pain? 

This IPD meta-analysis will enable us to perform the 
main and secondary analyses on an individual per-patient 
basis, the clinically most relevant level of analysis. The 
results may have relevant clinical implications because con- 
ventional coronary angiography (CCA) exposes the patients 
to infrequent yet considerable risks, such as myocardial 
infarction, coronary artery dissection and ventricular tachy- 
cardia [25,26], which might be avoided using CT in selected 
patient populations with suspected CAD. In addition, there 
are also economic considerations because CT might also be 
more cost effective [27]. 

Methods/Design 

This is a study protocol for the collaborative IPD meta- 
analysis on coronary CT angiography in comparison to 
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conventional angiography as the gold standard, called 
Collaborative Meta- Analysis of Cardiac CT (CoMe-CCT). 
It was registered with the PROSPERO database [28,29] 
(registration number: CRD420 12002780). This collaborative 
IPD meta-analysis is exempt from ethical approval as the 
analysis as well as the results for publication only involve 
de-identified data and all individual studies that will be 
summarized have received local ethics approval 

Identifying eligible studies for CoMe-CCT 

Valid published and unpublished studies will be retrieved 
by directly contacting all corresponding authors from 
published studies on coronary CT versus CCA in 
patients with suspected CAD. This will be achieved by 
including the primary studies of a recent regular systematic 
review and meta-analysis by our working group [13], and 
performing an update of this review to identify most recent 
studies. A sensitive search strategy using the particular 
thesaurus terms of the specific databases in combination 
with free text word synonyms [2] will be used to search 
MEDLINE (via PubMed), EMBASE (via Ovid) and ISI Web 
of Science databases for updating the article pool of rele- 
vant studies (suggested search strategies for all databases 
are provided in Additional file 1). We will also search 
bibliographies of retrieved studies, systematic reviews and 
meta-analyses, and include results of abstracts presented at 
most recent congresses as suggested by clinical experts. 

Key inclusion criteria are a prospective study design; use 
of a contemporary CT scanner with at least 12 simultaneous 
detector rows; use of >50% diameter reduction as the cut- 
off criterion for significant coronary artery stenoses in CT 
and CCA; application of CCA in all patients, regardless of 
the results of CT; provision of individual patient results for 
CT and CCA (positive, negative, and nondiagnostic), which 
allow calculation of 2x2 (excluding nondiagnostic results) 
or 3x2 (including nondiagnostic results) tables [30]; articles 
published in English or German; and provision of basic 
patient characteristics (age, gender, angina type, heart rate). 

Studies with the following characteristics will be 
excluded: retrospective study design; overlap with other 
studies as indicated by the corresponding author. 

Study screening and selection will be performed by two 
independent investigators. Retrieved articles will be 
searched after duplicate papers (from MEDLINE, EMBASE 
and ISI Web of Science) are identified and eliminated. In a 
first phase, titles and abstracts will be scanned and, in 
accordance with the inclusion and exclusion criteria, either 
discarded or retained for further investigation. In a second 
phase, the remaining potentially eligible articles will be 
reassessed in depth using the full text. 

Methodological quality assessment 

Two investigators will independently carry out the data 
extraction (including quality assessment) from all the 



retrieved published studies based on the full text articles. 
Discrepancies will be resolved in a consensus meeting, 
or, if agreement cannot be reached, they will be resolved 
in consensus with a third investigator. To assess and 
quantify the inter-rater agreement between the two 
investigators in evaluating methodological quality using 
the QUAD AS tool [31], we will calculate the kappa 
statistics. The data extracted will be: general and detailed 
methodological study characteristics, characteristics of 
the study population, details of the CT technology used, 
and detailed reference standard (CCA) specifications. 

Collection of individual patient data 

An international steering committee of renowned 
experts from the field of noninvasive coronary CT angi- 
ography has been established to lead, oversee and repre- 
sent the CoMe-CCT project. The steering committee 
consists of four clinical experts for cardiac CT: Stephan 
Achenbach, Erlangen, Germany; Matthew Budoff, Los 
Angeles, CA, USA; Mario J Garcia, New York, NY, USA; 
Marc Dewey, Berlin, Germany, and one clinical expert 
for CCA: Michael Laule, Berlin, Germany. The steering 
committee is completed by the projects statistician Peter 
Schlattmann, Jena, Germany. 

Corresponding authors of identified eligible published 
studies will be contacted using mail including a cover 
letter detailing the objectives of the collaborative meta- 
analysis, background information on IPD meta-analysis, 
and a CD containing a data collection file for input of 
individual patient results for the project. The details of the 
obligatory and additional patient and study characteristics 
are shown in Tables 1 and 2. The filled out data templates 
will be send back to us by either mail on CD or, more 
conveniently, by email. Further communication will 
mainly be performed by email or phone. Corresponding 
authors will also be contacted about unpublished data that 
will be eligible for inclusion in the collaborative meta- 
analysis if the inclusion criteria are met. To further enlarge 
the clinical data pool, authors will additionally be asked to 
provide data from registries. These data will not be 
included in the primary analyses, which are only based on 
data from the prospective studies. 

Data management, security and validation 

The same two investigators who will perform the 
electronic database search and identify published eligible 
studies for the regular review will also collect and 
assemble the IPD provided by the investigators of 
published and unpublished studies. Data will be sent as 
email attachments to the coordinating investigator of 
CoMe-CCT in an anonymized and de-identified way 
(without directly identifying information, that is, record 
numbers, names, addresses and so on will be removed). 
Data will be accepted in any kind of electronic format 
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Table 1 Obligatory computed tomography technology and patient characteristics requested in the data file 

Computed tomography characteristics Patient and conventional coronary angiography characteristics 

Publication from which the data is derived Age and gender, type of symptoms (typical angina pectoris, atypical angina pectoris, 

chest pain, no pain, pain unknown) 

Results of CT (positive 3 , negative, nondiagnostic 6 ). Results of conventional coronary angiography (positive 3 , negative, nondiagnostic 6 ). 

Comment and explanation for nondiagnostic patients Comment and explanation for nondiagnostic patients 

a At least one >50% stenosis in one vessel; b no stenosis in any vessel and at least one vessel not fully evaluable. 



(for example, SPSS, Stata, Word document,) but, ideally, 
the investigators will provide the data using the 
individualized Microsoft Excel (Microsoft Corp., Redmond, 
WA, USA) spreadsheets made available to them. The 
original data collection files sent by the authors will be kept 
in their original version and will be saved on a password- 
protected server behind the firewall in order to ensure 
security. Only the coordinating laboratory of CoMe-CCT at 
Charite will have direct access to the individual data. Both 
investigators will each perform data validation using a copy. 
To ensure quality of the data, they will independently 
check the received data sets for data entry mistakes and 
consistency and will compare the sum of the individual 
patient results received with the published summary 
results from the primary studies. Differences will be 
discussed and settled in consensus, if possible. They will 
contact the providing corresponding author in case of 
discrepancies that should be resolved. 

Outcome measures and statistical analysis 

An extension of the exact binomial rendition [32] of the 
bivariate mixed-effects regression model developed by 
van Houwelingen et al. [33,34] for treatment trial meta- 
analysis and modified for synthesis of diagnostic test 
data [35,36] will be used for data synthesis. This bivari- 
ate statistical model does not transform pairs of positive 
and negative predictive values into a single indicator, but 
preserves the two-dimensional nature of the data taking 
into account any correlation between the two. Given 
that we are dealing with individual data, a three-level 
bivariate random-effects model will be applied. On the 



first level of the model, a binomial distribution will be 
assumed for the true positives among the test positives 
and the true negatives among the test negatives. On the 
second level, the variability between patients within a 
certain study will be modeled. Finally, on the third level the 
variability between studies will be taken into account. 

The primary covariate of interest is the pretest likelihood 
of CAD. It will be calculated using a validated prediction 
tool [10] based on the original Diamond and Forrester 
model for risk calculation [7]. Using this tool, the pretest 
likelihood of a patient based on age, gender and clinical 
presentation can be estimated. Furthermore, the influence 
of covariates such as heart rate and body mass index of 
patients as well as the number of CT rows, and the admin- 
istration of beta blockade and nitroglycerin, will be taken 
into account on the individual patient level. Model selection 
will be based on the likelihood ratio test and Bayesian 
Information Criterion. 

Based on this model, mean logit positive predictive 
values and negative predictive values with their standard 
error and 95% confidence intervals, estimates of the 
between-study variability in logit positive and negative 
predictive values, and the covariance between them will 
be estimated. These quantities will be back-transformed 
to the original scale to obtain summary positive and 
negative predictive values. Using this data, a statistical 
prediction for a new cohort will be performed following 
the ideas presented by Skrondal and Rabe-Hasketh 
[37,38], Estimation will be done with SAS 9.2, Proc 
GLIMMIX and STATA 11, using the package GLLAMM. 
In addition, for performing a sensitivity analysis, we will 



Table 2 Additional computed tomography technology and patient characteristics requested in the data file 



Additional computed tomography characteristics 


Patient characteristics and additional tests 


Effective radiation dose 


Weight and height 


Type of electrocardiographic gating 


Derived body mass index 


Number of detector rows used 


Calcium score, heart rate during scanning, presence of cardiac risk factors 




(hypertension, diabetes, hyperlipidemia, current or former smoker, positive 




family history, prior myocardial infarction) 


Beta blockade (type, route, dosage) 


Results of rest electrocardiography 


Nitroglycerin (type, dosage) 


Results of stress electrocardiography 


Contrast agent (type, concentration, flow, amount) 


Results of stress echocardiography 


Breath hold duration 


Results of stress scintigraphy 
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also use WinBUGS [39]. If necessary, macros will be 
written to facilitate the use of three-level generalized 
linear mixed-effects models. 

Based on the model described above, mean logit 
sensitivity and specificity with their standard error 
and 95% confidence intervals, estimates of the between- 
study variability in logit sensitivity and specificity, and the 
covariance between them will be estimated. These 
quantities will be back-transformed to the original 
ROC (Receiver Operating Characteristic) scale to obtain 
summary sensitivity, specificity and the diagnostic odds 
ratios. The derived logit estimates of sensitivity, specificity 
and respective variances are then used to construct a 
hierarchical summary ROC curve for CT with summary 
operating points for sensitivity and specificity on the 
curves and a 95% confidence contour ellipsoid. 

Here the individual patient-specific covariates such as 
type and severity of symptoms, body mass index, beta 
blockade and sublingual nitroglycerin will be used in the 
regression analysis. This serves as tool to identify 
spectrum bias and clinically relevant influential factors. 

For performing sensitivity analysis, the analysis will be 
redone by leaving out one study. The generalized linear 
mixed model is known to be sensitive to starting values. 
Thus an additional analysis with Bayesian methods using 
PROC MCMC, SAS 9.2 will be performed. 

Discussion 

We believe that this collaborative multi-centric and 
multi-continental IPD meta-analysis will have an important 
impact on the clinical management of patients with 
suspected CAD as it should provide an answer to the 
question of which patients benefit most from coronary 
CT angiography. Furthermore, additional factors that 
influence the diagnostic performance of CT in comparison 
to CCA can be identified in this meta-analysis. This IPD 
meta-analysis also holds the potential to analyze and 
compare the predictive value of cardiac CT and CCA for 
subsequent major adverse cardiovascular events. Finally, 
the CoMe-CCT may facilitate the adequate selection of 
patients for cardiac CT by estimating the pretest likelihood 
for disease and predicting the diagnostic performance for 
individual patients that should ultimately help to avoid 
unnecessary examinations and thus decrease the use of 
scarce health care resources in the future [40]. 

Trial status 

Currently, in the course of performance of the IPD 
meta-analysis, we are in the state of data collection and 
data validation. No statistical analysis has yet been 
performed. The update search still has to be performed 
to be able to identify most recent articles and to contact 
the corresponding authors. At the time of study protocol 
submission, 54 single data sets from 40 authors and 



including more than 5,500 IPD comparing CT and CCA 
have been acquired. 

Additional file 



Additional file 1: Search Strategies for MEDLINE, EMBASE and ISI 
Web of Science. 
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